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Abstract 

This article reports findings from a study conducted with students in three sections of a Health Psychology 
course that replaced a traditional textbook with open educational resources (OER) as the primary course 
material. The purpose of the study was to learn how OER impacted students. Data were collected in Fall 
2015 with students from New York City College of Technology (City Tech), of the City University of New 
York (CUNY), a comprehensive college located in Brooklyn. Students were assigned the OER by their course 
instructor, who developed it as part of a library funded OER pilot initiative. Two research instruments were 
employed: one-on-one interviews and short surveys. Both interview and survey items asked students about 
how they engaged with the OER as their primary assigned course material. They shared feedback about the 
overall organization of the OER, ease of use, methods used to access the OER and complete coursework, 
benefits and challenges, and differences and similarities to using a traditional print textbook. 

Findings indicate that most students were able to access the OER more easily than traditional textbooks 
and responded positively to the variety of learning materials and assignments the OER assembled. Most 
students reported that course readings were equal to or better than traditional textbooks and would be 
willing to register for a course offering a similar resource in the future. A small amount of students reported 
minor usability issues. Also, few students had difficulties obtaining technology necessary to access the OER. 

Keywords: open educational resources, oer, open education, open digital pedagogy, assessment, 
evaluation, student feedback 


Introduction 

The purpose of this study was to learn how the use of open educational resources (OER) in place of 
traditional textbooks has impacted students’ academic experiences. Research was conducted with students 
in three sections of a Health Psychology course at New York City College of Technology, a comprehensive 
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college located in Brooklyn, New York. Through interviews and surveys, students were asked what methods 
they used to complete course assignments from the OER, how the OER was organized, and what it was like 
to navigate. They were also asked about the benefits and challenges of using the OER, and how it compared 
to a traditional textbook. The feedback collected in this study was intended to help assess and improve the 
OER program at the College, particularly the faculty development component of the initiative, and to 
encourage faculty reflection and foster best practices for developing, revising, and teaching with OER. 
Topics of value to discuss with faculty include student technology access and usage, user experience and 
navigation of OER sites, preferences for types of learning materials, and more. The findings from this study 
also facilitate an exemplar of positive and critical feedback about aspects of the Health Psychology OER that 
can provide concrete examples for instructors developing OER in the future. 

Open educational resources are “teaching, learning, and research resources that reside in the public domain 
or have been released under an intellectual property license that permits their free use and re-purposing by 
others” (Hewlett, 2016, para. 7). In addition to being cost free, open educational resources are released in 
the public domain, or have a Creative Commons license which allows the creator to retain their copyright 
and decide how their work gets used, shared, and remixed by others (Creative Commons, 2016). In contrast 
to textbooks, OER can be frequently updated, customized, or reworked by instructors in local contexts. 

City Tech's OER Initiative 

City Tech was motivated by the advantages that OER present for reducing or obviating the price of textbooks 
and providing more consistent access to curricular materials, particularly in light of the population City 
Tech serves. The college is one of 24 colleges that comprise the City University of New York (CUNY), the 
United States’ largest urban public higher education institution that is legislatively mandated as the “vehicle 
for the upward mobility of the disadvantaged in the City of New York” (City University of New York [CUNY], 
2016, para. 4). Pitt, Ebrahimi, McAndrew, and Coughlan (2013) contend that providing easy access to and 
customization of OER is especially important among community college and commuter student 
populations who often juggle a number of different responsibilities, and come from a variety of 
backgrounds, and levels of academic experience. Such attributes are consistent with the City Tech student 
population. More than 85% of City Tech students receive financial aid and 44% are among the first 
generation to attend college. The student population is diverse: 32% of students are Hispanic or Latino, 
30% are Black or African American, and 20% are Asian. Fifty percent of students at City Tech report a 
household income of less than $25,000 per year and about 10% earn less than $10,000 per year (CUNY 
Office of Institutional Research and Assessment, 2014). 

In addition to attending college, half of City Tech students work: 21% work 35 hours or more per week, and 
27% work between 21 to 34 hours per week. Students’ most commonly reported earning money in order to 
pay for living expenses, tuition, and fees. Since City Tech students devote much of their time earning a living 
and fulfilling non-academic responsibilities in addition to college, 43% report that work responsibilities 
affect their academic performance (CUNY Office of Institutional Research and Assessment, 2014). 
Moreover, Regalado and Smale’s (2014, 2015) ethnographic study of CUNY students’ academic and 
technological behaviors reveals the importance of understanding how access to technology, commuting, 
and other day to day realities converge with academic careers in meaningful ways. Their findings were 
encouragement for this study to include questions about students’ access and use of online based course 
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materials particularly because faculty interviewed in their study voiced concerns about students’ technology 
skills in academic settings. 

The OER initiative at City Tech is run out of the College’s Library and launched in Fall 2014. The pilot 
initiative awarded stipends for faculty to replace textbooks with a curated resource composed of openly 
licensed educational materials, as well as library digital subscription content, and other cost free 
educational materials. The initiative was modelled after similar programs launched at UMass Amherst and 
Temple University in the early 2010s (Bell, 2014; UMass Amherst Libraries, 2016). Completed OER were 
hosted on the OpenLab, City Tech’s unique WordPress powered open source collaborative platform for 
teaching and learning. Data were collected in December 2015 at New York City College of Technology (City 
Tech). 

Study Significance 

This study was intended to engage students and foster a more inclusive and equitable platform on which to 
consider big pedagogical initiatives, in this case the creation/adoption of open educational resources as the 
primary curricular material in lieu of traditional textbooks. It aims to broaden existing discourse on OER 
adoption beyond methods often used to measure efficacy, by instead foregrounding dialogue from the 
learner's perspective, in line with the Freirian philosophy of working with students as involved subjects, not 
objects (Freire, 1996). The parameters of the OER program, and the attributes of the OER created, are 
contextualized to the institutional setting and students, which informs the data collected in this study. 

There are a few distinctions in the City Tech OER initiative worth mentioning. First, the initiative is not an 
open textbook adoption program; rather, it supports faculty creation of OER by curating multiple open and 
alternative educational materials that constitute a collection of course materials. Another unique feature of 
the City Tech initiative is the OER hosting platform. OER are accessed via the City Tech OpenLab, an open 
teaching and learning system (as opposed to closed systems like BlackBoard) where it is possible for the 
City Tech community to interact with each other and the public (OpenLab, 2016). To support the open 
pedagogy philosophy, the OpenLab has several out of the box, mobile optimized environments for students 
and faculty to create their own websites including portfolios, projects, and clubs. Faculty have additional 
authorization to create course sites that can be open to the public, thus accessible to students after their 
City Tech careers end. 

Features of the Health Psychology OER 

In this study, students shared their experiences with and assessment of the OER in their interdisciplinary 
Healthy Psychology course. This course was particularly suitable for the OER initiative because the 
curriculum tackles contemporary issues from various disciplines such that the instructor felt there was no 
suitable textbook on the market. 

The learning materials that were collected and posted together on a course site are a combination of existing 
OER, instructor generated OER, and external sites that are linked out to, including library subscription 
materials, and free to view materials that are not Creative Commons licensed. Learning objects consisted 
of video lectures, news, and scholarly articles, PowerPoint slides, and readings from Boundless, a for-profit 
company, which at the time the study was conducted, provided free textbook content with customized quiz 
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and assessment activities on a digital platform. Since the course site includes all required assignments and 
readings, the syllabus, due dates, etc., it is distinct from a traditional textbook because everything is in one 
online location - effectively a full course “hub” housed on a WordPress powered site on the City Tech 
OpenLab. 


"Critical" Health Psychology PSY3405 
D919, Fall 2015 

An Interdisciplinary Course on Race and Health 



Course Profile Home Assignment Outline Assignments Course Requirements Lectures 


New York Times: Patient’s Voices Reference Articles Syllabus Readings Reading Outline 


Figure i. Detail of the Health Psychology landing page. 
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Figure 3. Detail of the Assignments (by subject) drop down menu. 


160 










What Impacts do OER Have on Students? Students Share Their Experiences with a Health Psychology OER at New York City College of Technology 

Cooney 


AT&T 4G 7:16 PM -f $ ■"! ’ 

a openlab.citytech.cuny.edu 



Course Profile Home 
Assignment Outline Assignments 
Course Requirements Lectures 
New York Times: Patient’s Voices 
Reference Articles Syllabus Readings 
Reading Outline 


About 

© 

top 

Welcome to PSY3405 Section D919’s OpenLab 

Figure 4. Detail of the mobile version view. 


Literature Review 

Current literature discussing the impacts of OER programs in U.S. postsecondary institutions is growing 
but diminutive. The literature reveals that a frequent method of assessing these programs, particularly the 
efficacy of the OER, employs student performance metrics such as exam scores, pass rates, GPA, final 
grades, assessment instruments specific to disciplines, and withdrawal rates and credits earned. Student 
and faculty surveys or questionnaires are another method commonly used to assess OER programs. Rarely 
are interviews and focus groups conducted to learn about student perceptions of, and experiences with OER 
in depth. 

Hilton’s (2016) recent synthesis of literature assessing OER programs published through September 2015 
discusses 16 studies, nine of which utilized student performance measures to explore the efficacy of the OER 
in comparison to formerly assigned textbooks, with learning outcomes and course enrollment trends. 
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Hilton analyzed and critiqued the rigor of the methods used in these studies and cited flaws and limitations 
in research design. These included suboptimal sample sizes for control and treatment groups, and multiple 
uncontrollable variances. Hilton concluded that despite the varying results of the nine studies, students’ 
learning did not decrease with OER, and their performance remained the same. 

Surveys are also a frequent component of OER assessment studies that seek to learn about students’ 
perceptions of OER. At the time of writing this, I am aware of eight published articles (Bliss, Hilton, Wiley, 
& Thanos, 2013a; Bliss, Robinson, Hilton, & Wiley, 2013b; Feldstein, Martin, Hudson, Warren, Hilton, & 
Wiley, 2012; Hilton, Gaudet, Clark., Robinson, & Wiley, 2013; Lindshield & Adhikari, 2011; Lindshield & 
Adhikari, 2013; Pitt et al., 2013; Petrides, Jimes, Middleton-Detzner, Walling, & Weiss, 2011) and two 
institutional reports (Affordable Learning Georgia, 2015; TideWater Community College, 2015) that 
conducted student surveys. Two studies evaluating open textbook adoptions: the multi-institutional Project 
Kaleidoscope initiative (Bliss et al., 2013a, 2013b), and the open statistics textbooks project at De Anza 
College in California (Illowsky, Hilton, Whiting, & Ackerman, 2016) employed questionnaires. In contrast 
to this study, most of the above mentioned research assessed open textbooks with larger student samples. 
Trends in student responses indicate that OER users were satisfied with the cost savings afforded (Bliss et 
al., 2013a; Hilton et al., 2013; Petrides et al., 2011). Of the surveys and questionnaires that asked students 
to compare the OER to a traditional textbook, the majority of students found the OER to be comparable 
(Bliss et al., 2013a, 2013b) or preferred the OER (Feldstein et al., 2012; Illowsky et al., 2016; Lindshield & 
Adhikari, 2013; Petrides et al., 2011; TideWater Community College, 2015). Most respondents also reported 
they would recommend the OER to classmates (Hilton et al., 2013; Pitt, et al., 2013; TideWater Community 
College, 2015). 

The studies had very few negative student responses. Of the two isolated groups that received negative 
feedback in the textbook transformation grants in Georgia, they both cited problems with usability, 
organization, and the writing style of the resources included in the OER (Affordable Learning Georgia, 
2015). Writing style was also a theme expressed in feedback among students at De Anza College who 
assessed their open textbooks by considering the level of comprehensiveness, readability, and applicability 
(Illowsky et al., 2016). Hilton’s (2016) literature review indicates that a minority of students across seven 
of the studies mentioned above found the OER to be inferior. 

Several other findings from the above studies are also worth noting. Of those surveyed, 72% of students 
who used an online Flat World Knowledge textbook in Virginia State University’s business program 
preferred it because of its enhanced portability over textbooks and 62% of students agreed the Flat World 
Knowledge textbook helped them to engage more with their course than a traditional textbook. The 
researchers acknowledged that students may have had biased preference toward the OER since it was free 
(Feldstein et al., 2012). Students who used an open version of a textbook called a flexbook in a Human 
Nutrition course at Kansas State University also preferred the ability to access multiple formats of the text 
such as PDF and Google documents versions (Lindshield & Adhikari, 2013). 

To date, I am aware of only two other studies that interviewed or conducted focus groups with students. 
Petrides et al. (2011) studied the impacts of the Community College Open Textbook Project (CCOTP) in 
2008, a program conducted with community colleges throughout the U.S. The second study that conducted 


162 



What Impacts do OER Have on Students? Students Share Their Experiences with a Health Psychology OER at New York City College of Technology 

Cooney 


interviews and focus groups, reported their survey and student performance analysis and did not discuss 
results from focus groups and interviews (Pitt et al., 2013). The findings of Petrides et al. (2011) support 
that students most commonly noted cost savings and easier access to the materials as benefits. Portability 
was a deciding factor in using the open textbook for their course according to 30% of student respondents. 
Both focus group and survey data supported the researchers’ findings that “students revealed they 
incorporated the open textbook into their current learning practices, specifically with regard to technology 
use” (Petrides et ah, 2011, p. 45). Their analysis also indicates that 77% of students preferred using print 
versions (hard copy or loose prints) of the open textbook during class time. Despite this preference, a 
minority of students reported printing out either the entire textbook or portions of it. They also report that 
students found the open textbook “supported increased interactivity with course materials” (Petrides et ah, 
2011, p. 45)- 


Methods 

To understand how using an OER impacted students, interviews and surveys were conducted upon 
receiving approval from the college Institutional Review Board, and obtaining appropriate consent forms 
from participants. Participant eligibility was limited to students enrolled in the Health Psychology course 
whose section was assigned the OER created by their instructor in the OER pilot program, who also taught 
all three sections. Surveys and interviews were conducted anonymously. Both instruments were designed 
to stimulate dialogue about usability, and the curricular materials comprised in the Health Psychology OER. 

Two students from each of the three Health Psychology course sections were recruited for interviews on a 
voluntary first-come, first-served basis. A total of six students participated in the semi-structured one-on- 
one interviews lasting approximately 45 minutes each and were compensated for their time with $15 
MetroCards. Participants were read and provided with a print copy of standard instructions about the 
interview process to maintain the consistency of each interview (Creswell, 2009) and each of the 12 
interview questions were read aloud to students via script. It was also stated that both the participant and 
the interviewer could ask questions to clarify and follow up as needed. This interview format provided the 
flexibility to pose open-ended questions that counterbalanced close ended survey items in order to glean 
more nuanced and detailed perspectives from students in their situated learning environments. 

Interview items asked where, with what devices, and how students used the OER to complete coursework. 
Students were also asked for their feedback on the usability of the OER, their perspectives on the resource 
as their primary course material, and how using the OER compared to a traditional textbook. 

There were a total of 86 students eligible to complete the survey from three sections of the Health 
Psychology course, and there were a total of 67 respondents. The aim of the survey protocol was to capture 
as much student feedback as possible to compare with student interviews. Several preceding studies were 
consulted (Feldstein et al., 2012; Hilton et al., 2013; Bliss et al., 2013; Affordable Learning Georgia, 2015) 
that asked similar questions in students surveys. Additionally, the OER Research Hub’s shared list of survey 
questions, and some online learning literature focusing on usability and instructional design (Young & 
Hoerig, 2013; Lee, 2014) were also consulted for additional guidance on survey items. Paper surveys were 
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distributed to students at the end of each class section. I provided students with a brief description of the 
research project and the purpose of the survey, and reminded students that the process was entirely 
voluntary, anonymous, and was unrelated to their course or grades at City Tech. 

The survey consisted of 12 questions that addressed demographics, and whether students had prior 
experience with the OpenLab. Another item asked about routine methods of accessing traditional 
textbooks. Remaining questions addressed the OER in relation to accessing the resource, completion of 
coursework, usability, navigation and ability to locate materials, quality, and students’ overall evaluation of 
the OER, especially compared to a traditional textbook. Likert scales captured student responses with 
frequencies, levels of agreement, likelihoods, and levels of difficulty. There were also multiple-choice 
questions, some of which included an option for open ended responses. 

Research Design Challenges 

Nomenclature was a challenging aspect of creating the research protocols. Students were not expected to 
be able to define or be familiar with open educational resources and the acronym: “OER.” Therefore, in both 
the survey and interview protocols, these terms are omitted. Instead, the OER are described in plain, 
discernable language. For this study, it is appropriate to refer to the Health Psychology resource as an OER, 
but it was consistently referenced to students as the Critical Health Psychology course site and discrete 
learning objects within the site were referred to as “course materials” on the Critical Health Psychology 
course site. 


Results and Discussion 


Demographics 

Demographics indicate that students surveyed in three sections of Health Psychology were diverse in their 
levels of academic experience and disciplinary tracks. Students represented a wide distribution with fifteen 
distinct majors reported among the 67 survey respondents. Computer Systems Technology - a baccalaureate 
program - was the most frequently reported major with 24 (36%) students. The next most prevalent majors 
were Hospitality Management and Nursing; each with eight (12%) students. The third most frequently 
reported major was Human Services, with five (7%) students. The remaining 22 were divided among 11 
different majors, with one to three students per major. Students who were interviewed studied Human 
Services, Computer Systems Technology, Radiologic Technology, Electrical Engineering Technology, and 
Nursing. The majors reported indicate that students from both baccalaureate and associate degrees were 
well represented in the study sample. 
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Table 1 

Distribution of Student Majors (11=67) 



Major/Program 

Degrees 

offered 

Percent of 
students 

Student 

response 

Applied Mathematics (Finance) 

BS 

1% 

1 

Architectural Technology 

AAS & B Tech 

1% 

1 

Biomedical Informatics 

BS 

1% 

1 

Construction Management & Civil 
Engineering Technology 

AAS 

3 % 

2 

Computer Information Systems 

AAS 

3 % 

2 

Computer Science 

AS 

3 % 

2 

Computer Systems Technology 

B Tech 

36 % 

24 

Electrical Engineering Technology 

AAS 

4% 

3 

Entertainment Technology 

B Tech 

3 % 

2 

Health Services Administration 

BS 

3 % 

2 

Hospitality Management 

AAS & B Tech 

12% 

8 

Human Services 

AAS & BS 

7% 

5 

Liberal Arts 

AA&AS 

4% 

3 

Nursing 

AAS & BS 

12% 

8 

Radiologic Technology 

AAS & BSRS 

4% 

3 


Of those surveyed, 80% of students reported having taken between three to eight semesters of college - this 
was the same range reported by students in the interviews - and the remaining 20% of students completed 
between 9 and 14 semesters. Students also indicated whether they used the OpenLab platform prior to this 
course: 38 (57%) students used the OpenLab prior to taking the Health Psychology course, and 29 (43%) 
students had not. 

Access and Technology 

When surveyed about what methods they used to access their assigned readings from the OER, 70% of 
students reported reading on their personal laptop or computer all or most of the time. This is an interesting 
finding given the transient nature of urban commuter students, and the ubiquity of mobile phone usage. 
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Nevertheless, reading on a smartphone was the next most common method that 31% of students reported 
doing all or most of the time. 

Fifty six percent of students reported reading with a City Tech computer regularly (31%) to all or most of 
the time (25%). Students least frequently read by printing their readings out: 49% of students rarely to 
never printed readings, while only 16% printed all or most of the time. Half of the students interviewed 
mentioned not printing anything either. The second least common method of accessing the readings was 
via tablets: 40% of students said they rarely to never used tablets. The drop in tablet usage contrasts 
Regalado and Smale’s (2014) earlier reports that indicated a rise in tablet usage in 2011. 

Table 2 


Distribution of Student Responses Regarding How They Access the Assigned Readings From the Health 
Psychology OER (n=66) 



Answer options 
(select all that apply) 

All or most of the 
time 

Regularly 

Occasionally 

Rarely/ 

Never 

I print them out. 

16% 

11% 

24% 

49 % 

I read them on a smart phone. 

31% 

13% 

30% 

26% 

I read them on a personal laptop or 
computer. 

70% 

23% 

2% 

5 % 

I read them on a City Tech 
computer. 

25% 

31% 

16% 

27% 

I read them on a tablet. 

18% 

22% 

20% 

40% 


Printing 

Respondents indicated why they printed readings if they had done so. Of those who responded, the majority 
reported printing so they could take notes on the page (65%), while the second most common reason 
reported was a preference for paper (41%). The response rate for this question was 46. In the prior question 
there were 66 respondents. Based on this deviation in number of respondents, one could infer that about 
30% of students found the question not applicable to them, which suggests roughly that percentage of 
students did not print at all. 
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Table 3 

Distribution of Student Responses to the Question: “If You Printed Out Any Readings for the Course , Why 
Did You Print?” (n=46) 



Answer options (select all that apply) 

Percent of 
students 

Student 

response 

I printed because I prefer paper. 

41% 

19 

I printed because it is hard to access the readings online 
when I'm off campus. 

17% 

8 

I printed because it is hard to access the readings online 
while I'm at City Tech. 

4% 

2 

I printed because I wanted to take notes on the page. 

65% 

30 

Other. 

15% 

7 


Completing Assignments 

While most of the students interviewed in this study mentioned using a mobile phone to complete readings 
and quizzes online from time to time (out of convenience, or to maximize their time), two students preferred 
completing course assignments with computers on campus. The remaining four students used a 
combination of smart phones, tablets, laptops, and desktops. During interviews, students shared how they 
accessed the OER on a daily basis. 

Depends on the day, and like what I’m doing. I live far out in Brooklyn so my train goes outside, so 
sometimes I’ll use it on my phone while I’m on the train...I have a Kindle so I can access it from my 
Kindle, or if I’m home I just use my laptop. 

-Student 

The device they used and their work location depended on multiple variables: schedule and type of 
assignment, preference for working environment, and level of convenience. 

[I use the] school computer cuz I don’t like using the wifi here cuz it’s like a whole process, but 
when, sometimes I’m on the bus I use my actual data from my phone. And then if I’m home I use 
my laptop or my actual...[desktop]. 

-Student 

As the findings from Regalado and Smale’s (2014) earlier study demonstrated, students’ abilities to engage 
in academic work hinged on access to computers at home or on campus. All of the students interviewed - 
with one exception - mentioned routine use of school computer labs to work on assignments. The student 
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who did not mention using the computer labs preferred working off campus, and found it easier to work 
outside of home, so Starbucks was her typical site for academic work. One student completed most of his 
work at the Library, citing it was optimal to working at home, while a second student preferred completing 
readings at home, and a third preferred writing assignments at home. 

Challenges Accessing the OER 

Students most frequently cited difficulties with accessing the OER via campus wifi (27%), and seven (10%) 
students had difficulty accessing wifi off campus. Though a small percentage of the total sample, this 
presents a notable barrier to accessing primary course materials, especially given that wifi on campus is 
unreliable. Three (4%) students reported trouble finding a device to access the OER on campus and two 
(3%) reported difficulty finding a device to access the OER off campus. Although these numbers are small, 
they are an indicator that some of our students face tremendous access barriers that should be mitigated 
with design solutions, and improved services, and infrastructure. 

Students also reported difficulties associated with usability that will be discussed in the next section. Due 
to the design of the survey item, there is no indication whether reported challenges were persistent or 
isolated, so it is unknown whether students overcame them. 

Table 4 

Reported Challenges Associated with Accessing Materials from the OER (n=63) 


Answer options 

Student response 

No challenges 

26 

Finding wifi to access the course site on 
campus 

18 

Knowing where to find the resources 

15 

Technological problems with 
downloading resources 

12 

Finding wifi to access the course site off 
campus 

7 

Taking notes on the resources 

5 

Printing the resources 

4 

Finding a device to access the course 
site on campus 

3 
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Organization and Usability 

Students who were interviewed commonly referred to the site as well organized and easy to use. Other 
words students used to describe the OER were helpful, user-friendly, and a “guideline to the semester.” 
They also mentioned how the OER is categorized and has tabs or pages for different course components. 
The “due dates” page on the OER was most frequently mentioned when students described how the OER 
site was organized. 

One student voiced that there was an initial learning curve to understand how to navigate the OER. Despite 
initial differences in comfort levels and familiarity with the OpenLab platform, all of the students 
interviewed found it easy to gain proficiency with the OER. 

When asked how they would describe the OER to a new user, one student said people would adapt to it right 
away because it is comparable to technology we routinely use, likening it to social media. 

Log on... navigate through it. Wherever you need to go you can find it. Use the search option. 

-Student 

Students also provided recommendations for prospective users. One said they would encourage people new 
to the OER to click through everything on the site to get a feel for it and become comfortable navigating. A 
second student recommended printing out everything on the OER (the tabs/pages that comprise the site) 
so they know and have a copy of everything they need, including the list of due dates. A third student said 
that becoming familiar with the OpenLab platform by signing up for an account in advance would help. 

The survey also asked students what it was like to locate materials on the OER. Students responded 
positively by reporting that it was easy (52.2%) or extremely easy (35.8%) to find what they needed (totaling 
88%). Seven (10.4%) students had a neutral response, and one student found navigating the OER to be 
difficult. Paradoxically, a sizable minority of students (24%) also reported having trouble knowing where to 
find the resources (Table 4). It is unclear whether survey respondents were able to overcome the challenge, 
though students who were interviewed reported moving beyond initial navigational issues, and did not 
mention difficulties finding resources aside from one mention of a few broken links. 
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Table 5 

Distribution of Student Responses to the Question: “How Would You Rate Your Overall Ability to Find 
What You Need on the Critical Health Psychology Course Site?’’ (n=67) 



Answer options 

Percent of students 

Student response 

Extremely easy 

35-8% 

24 

Easy 

52.2% 

35 

Neither easy nor difficult 

10.4% 

7 

Difficult 

1.5% 

1 


Preferences and Impacts on Learning Habits 

All students who were interviewed mentioned their satisfaction with, and preference for having all the 
course materials in one place. Two students really appreciated having a full, up-to-date guide covering the 
semester long course in contrast to past professors who might assign work verbally or on a weekly basis. 
Both also mentioned liking the option to complete work ahead of time. 

It’s like having the whole course in the website so you know what to follow...You don’t need to wait 
for the professor to say, “next week this is due, and this is how you do it,” it’s just like, ok, so next 
week this is due, and it’s explains it to you on the OpenLab. 

-Student 

I like looking ahead sometimes... when I have free time... all the material and all the homeworks 
were available already. So I like that... if I wanna do something earlier than the due date I could. 

- Student 

Most students referred to the due dates as a “go to” that linked to all the supporting materials needed to 
complete their assignments. This functionality was a welcome departure from their descriptions of 
scheduling time to visit the library to make copies of assignments. 

Two students had contrasting responses to how the OER impacted their academic habits. They both 
responded in the context of having better access to the OER which created a new level of convenience. One 
student found this to be a positive influence because she was able to maximize and allocate time to complete 
work more efficiently. Instead of saving all of her course work for what she described as a “congested” 
window of six or eight hours when she was home, she could complete some work prior to arriving home, 
and free up more time to do other course work. 
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the little short times that I do have that are free, I actually utilize it in order to study, in order to 
read. While before, if I had the textbook, I wouldn’t do that cuz it’s kind of a hassle bring[ing] the 
textbook... I would just listen to music... [and] when I get home, I’ll read those chapters. 

-Student 

The other student found the added convenience almost made her procrastinate over completing the work 
more. This student also mentioned that she probably studied more frequently because of the accessibility. 

Survey respondents rated how each of the course materials supported their learning process, if at all. The 
responses were very positive, with the majority of students agreeing that each material supported their 
learning, suggesting that students liked using a variety of educational materials. Students reported that the 
required assignments and quizzes were most supportive of their learning process. 

Table 6 

Level of Agreement About Course Materials From the OER That Supported the Learning Process (n=66) 



Answer options 

Strongly 

agree 

Agree 

Neither 
agree or 
disagree 

Strongly 

disagree 

Available, 
but I don’t 
use this 

Student 

response 

Posting or reading 
comments on the 
course site supports 
my learning 
process. 

21% 

68% 

8% 

3% 

0% 

66 

Powerpoint lectures 
support my learning 
process. 

29% 

61% 

11% 

0% 

0% 

66 

Online video 
lectures support my 
learning process. 

33% 

52% 

9% 

3% 

3% 

66 

Scholarly articles 
support my learning 
process. 

27% 

50% 

14% 

6% 

3% 

66 

Links to other web 
resources support 
my learning 
process. 

26% 

59% 

12% 

2% 

2% 

66 

Required 
assignments 
support my learning 
process. 

38% 

54% 

8% 

0% 

0% 

65 
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Required quizzes 
support my learning 






65 

process. 

38 % 

54 % 

6 % 

2% 

0% 



OER Versus Textbook 

When asked how the OER was different from a traditional textbook, most of the students responded in the 
interviews by sharing features of the OER that they liked. Half of the students began their responses by 
saying the OER was easier to access and more convenient because everything was available in one place - 
the OER site. Half of the students also contrasted the OER with textbooks by mentioning the cost savings, 
and most of the students (four of the six interviewees) also shared satisfaction with not having to carry a 
textbook around. One student mentioned how the OER saved her from developing back pain - something 
that had happened with textbooks in the past. Other students shared their relief at not having to wony 
about transporting a textbook to and from class, to work, and elsewhere, which alleviated complications as 
they planned their academic and life schedules. A couple of students described the OER as helpful and a 
time saver. A student remarked on how the OER freed her from having to borrow the textbook from a 
library, something that was time consuming and created stress over worrying when the return date 
approached. Another student said that textbooks, while required, were rarely utilized in some courses 
anyway. 

Half of the students interviewed preferred the composition and format of the OER much more than a 
textbook. 

It made me more focused... you know, with the textbook -1 don’t like -1 can get kinda lazy about it. 
But I go on the computer and, it’s something that I like to do so it kind of motivates me more to do 
it. 

- Student 

Another student felt the quizzes that accompanied the readings on Boundless really helped her test her 
knowledge and review areas that she didn’t initially understand. 

[When using] a regular textbook, I would just read it and think in my mind, “Ok you understand 
everything.” But you actually didn’t because, I mean, no one forces you to read the questions [in the 
back] of the textbook. 

-Student 

A couple of students found the course readings to be far less overwhelming than a textbook because they 
could focus in on the important components. 

When you have the textbook, it’s a little bit overwhelming with all the information that’s in it. You 
have to look through it. You just feel like, ok, I have to read these chapters. So it’s overwhelming to 
see how much you have to actually do. But if you have- you print it out from the site, you say, ok, 
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this is what I have to do. This is the only thing I need to do. So it’s like you’re focused on only what 
you printed out. 

-Student 

One student mentioned her preference for watching videos, a learning habit she had developed 
independently by searching for YouTube videos related to other courses that did not integrate multimedia 
into the curriculum. 

Survey respondents also shared their perceptions of the quality of the OER readings. Eighty one percent of 
students found their quality to be somewhat to much better than a textbook. 

Table 7 

Response Rating on the Quality of the OER Course Readings Compared to a Textbook (11=67) 



Answer options 

Percent of students 

Student response 

Much better 

41.8% 

28 

Somewhat better 

38.8% 

26 

About the same 

16.4% 

11 

Somewhat worse 

1.5% 

1 

Much worse 

1.5% 

1 


Survey respondents also rated aspects of their course experience against a traditional textbook and the 
results were largely positive. Students were most positive about the OER increasing their satisfaction with 
the learning experience, and increased engagement with course lessons in comparison to a textbook. By 
contrast, 32% of students did not think using the OER instead of a textbook affected their grade and 35% 
reported it had no effect on their confidence. Twenty percent of students also reported that the OER did not 
impact their class participation or their interest in the course. 
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Table 8 

Level of Agreement About How the OER Impacted the Learning Experience (n=66) 



Answer options 

Strongly 

agree 

Agree 

Neither agree 
or disagree 

Strongly 

disagree 

Student 

response 

Using the course site 
instead of a traditional 
textbook: 
...increased my 
participation in class. 

38 % 

42% 

20% 

0% 

65 

...increased my interest in 
the course subject. 

29% 

52% 

20% 

0% 

66 

...increased my exposure 
to different ways of 
learning. 

41% 

47 % 

12% 

0% 

66 

...increased my 
satisfaction with the 
learning experience. 

42% 

47 % 

11% 

0% 

66 

...increased my 
engagement with the 
course lessons. 

41% 

42% 

15% 

2% 

66 

...improved my grade. 

38 % 

30% 

32% 

0% 

66 

...built my confidence. 

27% 

36 % 

35 % 

2% 

66 


Most survey respondents indicated they were likely to register for a course with a similar resource again 
(89), with 6 students (9%) neutral, and 1 student unlikely to do so in the future. 

Table 9 

Response for Likelihood Students Would Register for a Class Using a Resource Similar to the Health 
Psychology OER Instead of a Traditional Textbook (n=66) 



Answer options 

Percent of students 

Student response 

Extremely likely 

63.6% 

42 

Likely 

25.8% 

17 

Neutral 

9.1% 

6 
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Unlikely 

1.5% 

1 

Extremely unlikely 

0.0% 

0 


Criticism 

Students suggested that adding email notifications for important approaching deadlines, as BlackBoard 
does, would be helpful, making PDF versions of the readings would be an improvement, and that navigating 
to the OER site within the OpenLab platform could be easier. One student shared that there were glitches 
with using the drop down menu on the OER with smartphones and tablets, and that the OER had a few 
broken links. Overall, criticism of the OER was minimal. 


Conclusion and Recommendations 

Study findings indicate that students used a combination of digital devices to access the course readings, 
yet despite the prevalence of handheld and portable devices, readings and completion of assignments 
happened most frequently on personal and/or school laptops and desktops. The majority of students also 
preferred the OER to traditional textbooks. The latter is consistent with previous studies, but distinct in 
that most studies were about open textbooks as opposed to a curated OER site. Preceding studies also report 
that students were attracted to OER for the cost savings and ease of use and that was true for students in 
this study as well. Students in this study also voiced their satisfaction with not having to carry a textbook as 
they traveled throughout the day. 

Convenient access to course materials is an issue of particular gravity among underserved populations and 
urban commuter students who typically balance many commitments in addition to their studies. The 
manner students discussed their satisfaction with the convenience of having all the course materials 
organized pragmatically in one location suggests it is worth considering how the scaffolding of the course 
on the OER impacts students’ academic motivation and agency. Other preferences of students distinctly 
aligned to improved learning habits and enhanced teaching methodologies. For instance, students also 
expressed preference for more targeted and concise readings with accompanying quizzes that align with 
active reading and formative assessment. I believe students’ reports on the convenience of OER prompts us 
toward deeper analysis and inquiiy on this front, particularly relating to instructional design and 
integration of active and high impact learning methods. 

Educators and administrators are often in the position to make decisions that have lasting impact on 
students. In this study, I hoped to expose student voices to bring learners’ narratives in dialogue with 
pedagogical praxis. The emergence of OER is an important pedagogical shift that needs constructive 
channels of dialogue among learners and educators to be implemented effectively. OER practitioners can 
benefit from conducting research that yields in depth data by employing qualitative methods such as 
student focus groups and interviews that explore usability, learning habits, and technology access and 
proficiency. Students have valuable perspectives as demonstrated by the results of prior OER studies that 
incorporated student critiques (Petrides et al., 2011; Illowsky et al., 2016). The recommendations students 
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shared on how their open textbook could be become more interactive with annotation tools, audiovisual 
elements, and a venue for online dialogue with professors 

Limitations 

By design, this study was not meant to contribute statistical data; the inclusion of the survey had the primary 
function of informing interview data. While the strength of this study is its reliance on student perceptions, 
human subjects also introduce biases and their data cannot be externally validated. Additionally, 
recruitment for students interviewed could have favored those with more schedule flexibility and better 
time coordination skills, thus may have impacted the qualitative data sample and demonstrates that 
achieving a truly representative student sample is not possible. 
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